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3.44 Robustness, breakdown points, and 1-dimensional location M-estimators. 

[Parts of this section are more or less parallel to parts of Section 3.4. One difference is that 
here, breakdown points are defined in terms of finite samples. Eventually, this section and 
Section 3.4 will be merged into one or reorganized.] 

Let X = (Xi, Xn) and Y = (Fi, F„) be samples of real numbers. For j = 1, n 
let X =j Y mean that Xi = Yi except for at most j values of i. More specifically, for 
y = (yi, ...,yj) let X =j^y Y mean that for some integers v with 1 < ii < i2 < ... < ij < n, 
— Ur for T = 1., J and Yi = Xi \i i ^ for r = 1, j. The idea is that in the latter 
case, Xi are i.i.d. from a nice distribution like a normal and yr are errors or "bad" data. So 
the sample X contains n — j good data points and j errors. A robust statistical procedure 
will be one that doesn't behave too badly if j is not too large compared to n. 

"Breakdown point" is one of the main ideas in robustness. Let T = T{Xi, ...,Xn) be 
a statistic taking values in some locally compact metric space G such as a Euclidean space. 
The closure of a set A G Q will be denoted A. If G is a Euclidean space then a set A C G 
has compact closure if and only if sup{|x| : x e A} < oo. The breakdown point of T at X, 
or more specifically the finite-sample breakdown point, is defined as 

e*{T,X) = £*(T;Xi,...,Xn) = -max{j : {T{Y) : Y =j X} is compact}. 

71/ 

In other words s* (T, X) = j/n for the largest j for which there is some compact set K C G 
such that T{Y) E K whenever Y —j X. If e*{T, X) doesn't depend on X, which is often 
the case, then let e*{T) := e*{T,X) for all X. If G is a Euclidean space M'^, then the 
compactness condition in the definition of £* is equivalent to 

sup{|r(r)| : Y =j X} < +00. 

If a fraction of the data less than or equal to the breakdown point is bad (subject to 
arbitrarily large errors), the statistic doesn't change too much (it remains in a compact 
set), otherwise it can escape from all compact sets (in a Euclidean space, or by definition 
in other locally compact spaces, it can go to infinity). There are a number of definitions 
of breakdown point. The definition of finite-sample breakdown point as above is given in 
Hampel et al., 1986, p. 98, for a real- valued statistic. 

Since j in the definition is an integer, the possible values of the breakdown point 
for samples of size n are 0, 1/n, 2/?i, 1. A statistic with a breakdown point of is (by 
definition) not robust. Larger values of the breakdown point indicate more robustness, 
up to just less than 1/2. Finite-sample breakdown points > 1/2 are unattainable in some 
situations, e.g. Theorem 3.44.2 below. 

Recall the definition of order statistics: for a sample Xi, . . . of real numbers, let 
^(1) < ^(2) ^ ■ ■ ■ < -^{n) be the numbers arranged in order. 

Examples, (i) For the sample mean T = X = {Xi + ... + Xn)/n, the breakdown point is 

since for j = 1, if we let yi — * oo then X ^ oo (for n fixed). 

(ii) Let T — X(i) , the smallest number in the sample. Then the breakdown point of T is 
again since for j = 1, as yi — > — oo we have X(i) —>■ — oo. Likewise the maximum X(^) of 
the sample has breakdown point 0. 
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So the statistics X, X(^) axe not robust. Other order statistics have some 

robustness (for fixed finite n): 

Theorem 3.44.1. For sample size n, and each j = 1, ...,n, the order statistic T = 
has breakdown point e*{T) = ^ min(j — l,n — j). 

Proof. Take any sample X = We have inf {T(y) : Y =j X} = -oo (let 

yi, ...,yj all go to -oo). Likewise sup{T(r) : Y =n-j+i X} = +00 (let yi, ...,yn-j+i 
+00). Soe*{T,X) < ^mm{j -l,n-j). 

If Y =j-i X then the smallest possible value of Y^j) occurs when yi < Xk for all 
i and k and for at least one k such that Xk = ^{i), is not replaced, so > -^(i)- 
Similarly, if Y =n-j X the largest possible value of Y(j) satisfies Y(j) < So if 

r = min(j — l,n — j) and F =7. X, then < y(j) < -^(n) so is bounded and 
e*{T,X) = ^ min(j — l,n — j) as claimed. Since this is true for an arbitrary X, the 
theorem is proved. □ 

If J = 1 or n, the breakdown point of is as noted in the Examples above. If 
n is odd, so n = 2/c + 1 for an integer k, then the sample median X(^k+i) has breakdown 
point I ~ 2n ~ n- If = 2/c for an integer k, then the two endpoints of the interval of 
medians, Y(^kj and Y^fc+i), each have breakdown point 5 — ^- So any median has breakdown 
point at least | — ^— >|asn— >oo. From Theorem 3.44.1, no other order statistic has 
any larger breakdown point than the median, so £*(X(j)) < 1/2 for all j. This is typical 
behavior for interesting estimators. But, larger breakdown points are possible. If T has 
values in a compact set, then it trivially has breakdown point 1 by our definition. Or, let 
T = mhij \Xj\ for observations in R'^. Then one can check that T has breakdown point 
1-1. 

n 

For real-valued observations Xi,... a real-valued statistic T = T(Xi, Xn) 

will be called equivariant for location if for all real 9, and letting X = (Xi, . . . and 

x + e^{x^ + e,...,Xr^ + e), 

T{X + 9) = T(X) + d 

for all n- vectors X of real numbers and all real 9. 

For example, the order statistics X^^) and the sample mean X are clearly equivariant 
for location. 

Theorem 3.44.2. For any real-valued statistic T equivariant for location, the breakdown 
point is < 1/2 at any X = (Xi, ...,X„). 

Proof. Let the breakdown point of T at X be j/n. Then there is an M < cxd such that 

(3.44.3) |T(y)| < M whenever y =j X. 

Let 6 = 3M. Now W = Y + ior some Y with Y =j X if and only if W =j X + 9. 
Then T{W) = T{Y) + 9. So 

(3.44.4) \T{W) -9\<M whenever W =j X + 9, and then 2M < T{W) < AM. 
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But if J > n/2 there is a with W =j X and also W =jX + 9. For such a W, (3.44.3) 
and (3.44.4) give a contradiction, proving Theorem 3.44.2. □ 

Now, we'll consider breakdown points of 1-dimensional location M-estimators. Let 
be a real-valued function of a real variable which is odd (meaning 'i/j{—x) = —iIj{x)), 
nondecreasing, nonconstant, and bounded. Then ip{—t) < = 'ip{0) < 'ip{t) for alH > 
and i(^{—t) < < '0(t) for some t > since t/^ is nonconstant. We will have ip{t) A as 
t +00 for some A > 0. Examples of such functions t/j include the derivatives p'{x) of 
wide-sense Huber functions (as defined in §3.4), where such derivatives are defined, with 
suitable choices where they are not defined, specifically, '0(0) = in all cases, ip{x) := 
p'{x+) := lim.hiQ{p{x + h) — p{x))/h and ip{—x) := — (a;) for a; > 0. Then for location, 
the psi function of two variables is defined by ip{0, x) := ip{x — 6), which is nonincreasing 
in 9. Given a sample (Xi, . . . , X„), let 

9* := 9%Xi,...,Xn) := sup | ^ : ^V(^i-^)>0 

This is finite since the sum is < for ^ > and also < when 9 > -|- t for some t 
such that 'ip{t) > 0. Analogously, define 

9** := r*(Xi,... ,Xn) :=inf ^V(^i-^)<o|, 

which is also finite since the sum is > for ^ < -^(i). We have 9* < 9** because of 
the monotonicity of ip. Then a statistic T„ = Tn{Xi, . . . ,X„) will be an M-estimator of 
extended ip type if and only if 9* < < 9**. In order to have a unique estimator, the 
M-estimator defined by ijj and the sample will be defined, as for the sample median, by 

9 := 9{{xi,... ,xn)) := ^(r + r*)((xi, . . . , x^) ). 

It will shown that such estimators have the same (finite sample) breakdown points as the 
median, converging to 1/2 as n ^ oo and as large as possible. Consider also scale- adjusted 
M-estimators, where instead of Y^^=i fpi^i ~ ^) we have Y^^=i i^ii^i ~ ^'^'^ is a 

scale estimator, with nonnegative values. The resulting ^**, and 9 will be called 9*g^ 
9g*, and ^5 := {9* + 9**)/2 respectively. If 5 = 0, then by definition set 









> 9 


i;{{X,-9)/S) := 1 




X, 


= 9 








< 9 



It's easily seen that if 5 = then the M-estimator 9s based on the above definitions is 
exactly the median. 
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To get a particular choice of S, let M be the median of the sample, defined as X(fe_|_i') 
if n = 2k + 1 is odd, and (^(fc) + -^(fc+i))/2 if n = 2k is even. Let MAD denote the 
median absolute deviation, namely the median of \Xi — M|, and S = MAD/. 6745, where 
the constant 0.6745 is (to the given accuracy) the median of 1^1 for a standard normal 
variable Z, and thus, S estimates the standard deviation a for normally distributed data. 

The following fact and proof are adapted from Huber (1981), pp. 52-53. 

Theorem 3.44.5. Let be a function from R into R, which is odd, nondecreasing, 
nonconstant, and bounded. Then the M-estimator 9 defined by ijj has breakdown point 
^ — ^ if n is even and | — 2k if ^ is odd. The same holds for the scale- adjusted M-estimator 
9s where we consider '(/'((X^ — 9)/S) for the S just defined. 

Proof. As t — 00 we have 'i(^{t)^A. For < e < 1 there is a k < 00 such that i/^Ik) > 
(1 - e)A. Then ^lILi V'(^(i) - ^) < if - 6* < -At for j values of i where - e)A + 
{n — j)A < 0, or equivalently j > n/{2 — e). Now 9 > X^j) -|- k for at least j values of i 
is equivalent to ^ > -|- n. So we have 9** < X(j) -|- n where j is the smallest integer 
> n/{2-e). 

Now if Yi = Xi for at least j values of i = l,...,n, we have < X(^), so 
9**{Yi, ...,Yn) < ^(n) + 1"^ and 9** remains bounded above under the given conditions. 
Symmetrically, 9* stays bounded below. It follows that 9 stays bounded, so the break- 
down point of 9 is at least 1 — j/n. 

If z > n/2, then for some e > 0, z > n/(2 — e), so we can take j as the smallest integer 
greater than n/2. Then for n = 2m even, or for n = 2m + 1 odd, we have j = m + 1. It 
follows that the breakdown point of 9 is at least as large as stated for each sample size. 

^ is a location equivariant estimator, so its breakdown point is less than 1/2 by 
Theorem 3.44.2. So the breakdown point is no larger than for the median and is the 
same as for the median, proving the first statement in the theorem. 

Next, consider the scale- adjusted case and first, the breakdown point of S. If j is 
again the smallest integer > n/2, and Y =n-j ^ ^ so that Yi = Xi for at least j values of 
i, then <Yi< -^(n) ^or at least j values of i. Thus as noted above 

(3.44.6) < 

and if My is the median of Yi,...,y^, then < My < ^(n)- Also, \Yi — My\ < 

X(^) —^(1) for at least j values of i, so MADy, the median of |yi — My |, satisfies MADy < 
X(n) - X(^i^ and 

(3.44.7) Sy S{Yi, ...,Yr,) < Kx := (X(,) - X(i))/0.6745. 

On the other hand if y =fc X for /c > n/2 we can have My unbounded and also MADy 
unbounded. So the MAD and S have the same breakdown point as the median itself. 

It's possible that 5 = if there are enough tied observations. As noted above, the 
M-estimator 9s equals the sample median in that case. 

Returning to the case where j is the smallest integer > n/2 and Y =n-j X, take e > 
small enough so that j > n/{2 — e) and choose k accordingly. Then we will have 

n 

(3.44.8) J]V'((>"i-^)/^r)<0 
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if Yi — < —kSy for at least j values of i, in other words if — 9 < —nSy or 9 > 
Yj-j) + k,Sy- This does hold when Sy = 0: then if — ^ < for at least j values of z, we 
have by the definitions YJi=i '^iO'i - d)/SY) < -jA + (n - j)A = (n - 2j)A < 0. Now 
(3.44.8) holds if 6' > + kKx by (1) and (2). So 

r*(yi,...,yj <X(„) + «Kx. 

Symmetrically, we have 

r(Yi,...,Fn)>X(i)-«Kx. 

So 9{Yi, ...,Yn) remains bounded for Y =n-j X and the breakdown point of 9 is at least as 
large as for the median. If 5" = 0, this doesn't cause breakdown of the M-estimator because 
the median of a sample Yi, containing more than n/2 of the original observations 

Xi, must be between and -^(n) and so can't become unbounded. 

Since 9s is also location equivariant, its breakdown point is also < 1/2 and so equals 
that of the median, as stated in the Theorem. □ 

PROBLEMS 

1. Let be a positive integer and n = 4/c + 3. Let Xi, be a sample of real numbers 
with order statistics < ^(2) < ■■■ < ^(n)- Then the sample median is X(2A;+2)- 
Define to be the lower quartile and X(3fe+3) to be the upper quartile. (Quartiles are 
defined for sample sizes not of the form 4A; + 3 as combinations of adjoining order statistics 
around 1/4 and 3/4 of the way up the ordering.) The interquartile range is defined by 
IR = -^(3A;+3) — ^(/c+i)- What is its breakdown point? Does it depend on the sample? 

2. Suppose we want to estimate a scale parameter a with < cr < 00 (not necessarily 
a standard deviation). Each compact set in the parameter space is included in a set 

1/M <a <M. 

(a) For what samples Xi,...,Xn of real numbers is the median absolute deviation 
MAD in the parameter space? 

(b) If the observations Xj are all different, what is the breakdown point of MAD with 
the given parameter space? Hint: if the parameter space is [0, 00), so that is a possible 
value, the breakdown point is found in the proof of Theorem 3.44.5. 

(c) Suppose that r of the Xj are the same for some r — 2, ...,n and the rest are all 
different. Then what is the breakdown point of MAD at such a sample? 

3. For the wide-sense Huber functions p on p. 6 of Section 3.4, (a), (b) and (c), and 
if^i^x) = p'{x) for X ^ 0, '0(0) — fiiid the number A = lim^^^+oo i^{x) in each case. 

4. Show that for n = 2, the M-estimate for location defined by any ijj function satisfying 
the hypotheses of Theorem 3.44.5 is {Xi + X2)/2. 

5. Show that for the p function p{x) — \J\ + x"^ and corresponding '0 function '^(a;) = p'{x), 
we have 9* = 9** for any sample. 
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